Construction site safety monitoring and excavator activity analysis system

With the recent advancements in deep learning and computer vision, AI-powered construction machine such as autonomous excavator has made significant progress. Safety is most important section modern construction, where machines are more automated. In this paper, we propose a vision-based perception, activity analysis, safety monitoring system. Our perception system could detect multi-class humans real-time while estimating poses actions of excavator. Then, present novel analysis based on result. To evaluate performance our method, collect dataset using Autonomous Excavator System (AES) (Zhang et al., Sci Robot 6(55):eabc3164) including objects different lighting conditions with human annotations. We also method benchmark dataset. The results showed YOLO v5 detection model improved inference speed by 8 times (YOLO x-large) to 34 small) compared Faster R-CNN/YOLO v3 Proceedings 38th International Symposium Automation Robotics Construction 461 (ISARC), pp. 49–56. InternationalAssociation for (IAARC), Dubai, UAE (2021). https://doi.org/10.22260/ISARC2021/0009 ). Furthermore, accuracy models 2.7% size reduced 63.9% 93.9% small). experimental show that proposed action recognition approach outperforms state-of-the-art approaches top-1 about 5.18%. not only designed solid waste scenes, it can be applied general scenarios.